Packet 7 (KW addressing questions raised at Asilomar)

Last updated: 2019-05-08

This is Kara’s response to an email from Tanya from Asilomar (2019-05-07), post-conference, using Packet 7 data (with data from 2019-03-18). (Note: PDF printed portrait, 80% zoom, minimum margins.)

Questions from TML email:

1. Does DSE 4, 5, 13, 14, 15 [the least spiritual events] hang together with Spiritual Events and does porosity predict it?
2. Spiritual Events (three groups: 14, 15, 26, then 17, 18, 19, then 20, 21, then all these together)—do these correlate less with porosity than 1-13 from that scale?
3. Do these items from the Dulin scale (2, 3, 16, 17, the less personified events) correlate differently with spiritual events than the other items considered as a group?
4. Can we sum Martha Mary 1b, 1e, 2b, 2e, 3b, 3e and see whether these “pure” porosity events still predict spiritual events?
5. Are there highly religious people (as marked in demographics) who do not endorse porosity highly and the reverse? e.g., can we plot the how spiritual/religious against Martha Mary and also against Dulin?
6. Plot highly religious people against the DSE and against the Spiritual Events scale?

I’ll try to address each in turn!

Question #1: Focusing on the least spiritual experiences from Daily Spiritual Experiences scale

Tanya’s question: Does DSE 4, 5, 13, 14, 15 [the least spiritual events] hang together with Spiritual Events and does porosity predict it?

I’ll start by looking at correlations between the subset of “less spiritual” DSE items, the Spiritual Events total score, and the Porosity (Dulin) total score. I’ll also include the relevant correlations with the “more spiritual” DSE items and the DSE total score as points of reference.

Correlations with Spiritual Events:

• The subset of less spiritual DSE items was strongly correlated with the Spiritual Events total score (r = 0.58, 95% CI: [0.53, 0.61], p = 0).
• Numerically speaking, this correlation appears to be somewhat weaker than the correlation between the subset of more spiritual DSE items and Spiritual Events (r = 0.69, 95% CI: [0.65, 0.73], p = 0).

Correlations with Porosity:

• The subset of less spiritual DSE items was strongly correlated with the Porosity (Dulin) total score (r = 0.58, 95% CI: [0.53, 0.63], p = 0).
• Numerically speaking, this correlation appears to be somewhat weaker than the correlation between the subset of more spiritual DSE items and Porosity (Dulin) (r = 0.66, 95% CI: [0.61, 0.73], p = 0).

However, are these meaningful differences? One important thing to keep in mind is that the reliability of a scale (how strongly its items are correlated with each other) affects how strongly correlated the total scale can be with other measures. A rough rule of thumb is that the reliability of a scale sets an upper bound for the correlation between that scale and others. Since these two subsets of DSE are kind of ad-hoc (and definitely post-hoc) breakdowns of a pre-existing scale, this seems especially important to check here.

With this in mind, let’s check the reliability of the two subsets of the DSE scale, and look at how these correlations with Spiritual Events and Porosity (Dulin) stack up to the upper limit imposed by reliability… and just for fun I’ll throw in the correlations within each site as well.

The solid dots in these plots illustrate that the subset of less spiritual DSE items (center panel) formed a generally less reliable “scale” than the subset of more spiritual DSE items (rightmost). This has the important consequence of forcing the correlation between the less spiritual items and the Spiritual Events scale in some sense to be weaker.

Given this, what I’m interested in is whether the distance between the cross-hatch (the correlation coefficient between that scale and Spiritual Events, in the first plot; or between that scale and Porosity (Dulin), in the second plot) and the solid dot (the reliability of that scale) differs across the panels. If this distance were greater for the subset of less spirtiual DSE items, this would strengthen my belief that these items were meaningfully less strongly related to Spiritual Events/Porosity - but that’s not really what I see here in either case. Instead, if you think of reliability as an upper limit for correlation, the differences between correlations starts to look like an artifact of differences in reliability.

KARA’S CURRENT TAKE-AWAY: Both less spiritual and more spiritual DSE items were strongly correlated with Spiritual Events, and with Porosity. These relationships were somewhat weaker for the less spiritual DSE items, but I’m not sure these differences were meaningful—they might well be an artifact of underlying differences in reliability across the two subsets of DSE items.

Question #2: Focusing on different sets of spiritual events from the Spiritual Events scale

Tanya’s question: Spiritual Events (three groups: 14, 15, 26, then 17, 18, 19, then 20, 21, then all these together)—do these correlate less with porosity than 1-13 from that scale?

NOTES:

• In Packet 7, the Spiritual Events scale only included 23 items (no question #26), and I suspect you might have meant 16 instead, so I went with this for now. - I also want to point out that this grouping (1-13 vs. 14-21) leaves out questions 22-23.
• Given my concerns about reliability, I’m going to start with the coarsest grouping - comparing a 3-item “scale” with a 13-item scale could really obscure what’s going on here. We can come back to this and discuss more!

I’ll start by looking at correlations between SE items #14-21, SE items #1-13 and the Porosity (Dulin) total score. I’ll also include the relevant correlations with the SE total score as a point of reference.

Correlations with Porosity:

• The highlighted subset of SE items (#14-21) was strongly correlated with the Porosity (Dulin) total score (r = 0.49, 95% CI: [0.42, 0.55], p = 0).
• Numerically speaking, this correlation appears to be somewhat weaker than the correlation between the other subset of SE items (#1-13) and Porosity (Dulin) (r = 0.57, 95% CI: [0.52, 0.62], p = 0).

However, is this a meaningful difference? To repeat from Question #1, above: an important thing to keep in mind is that the reliability of a scale (how strongly its items are correlated with each other) affects how strongly correlated the total scale can be with other measures. A rough rule of thumb is that the reliability of a scale sets an upper bound for the correlation between that scale and others. Since these two subsets of SE are kind of ad-hoc (and definitely post-hoc) breakdowns of a pre-existing scale, this seems especially important to check here.

With this in mind, let’s check the reliability of the two subsets of the SE scale, and look at how these correlations with Porosity (Dulin) stack up to the upper limit imposed by reliability… and just for fun I’ll throw in the correlations within each site as well.

The solid dots in these plots illustrate that SE items #14-21 (rightmost panel) formed a generally less reliable “scale” than SE items #1-13 (center)—not surprising, since shorter scales are generally less reliable. This has the important consequence of forcing the correlation between items #14-21 and the Porosity (Dulin) scale in some sense to be weaker.

Given this, what I’m interested in is whether the distance between the cross-hatch (the correlation coefficient between that scale and Porosity (Dulin)) and the solid dot (the reliability of that scale) differs across the panels. If this distance were greater for SE items #14-21, this would strengthen my belief that these items were meaningfully less strongly related to Porosity - but that’s not really what I see here. Instead, if you think of reliability as an upper limit for correlation, the differences between correlations starts to look like an artifact of differences in reliability.

KARA’S CURRENT TAKE-AWAY: Both subsets of SE items were strongly correlated with Porosity. These relationships were slightly weaker for SE items #14-21, but I’m not sure these differences were meaningful—they might well be an artifact of underlying differences in reliability across the two subsets of SE items.

Question #3: Focusing on the less personified events from the Porosity (Dulin) scale

Tanya’s question: Do these items from the Dulin scale (2, 3, 16, 17, the less personified events) correlate differently with spiritual events than the other items considered as a group?

I’ll start by looking at correlations between the less personified Porosity items, the more personified Porosity items and the Spiritual Events total score. I’ll also include the relevant correlations with the Porosity total score as a point of reference.

__Correlations with Spiritual Events_:

• The less personified subest of Porosity items was moderately correlated with the Spiritual Events total score (r = 0.41, 95% CI: [0.33, 0.48], p = 0).
• Numerically speaking, this correlation appears to be weaker than the correlation between the other subset of SE items (#1-13) and Porosity (Dulin) (r = 0.58, 95% CI: [0.53, 0.63], p = 0).

However, is this a meaningful difference? To repeat from Questions #1-2, above: an important thing to keep in mind is that the reliability of a scale (how strongly its items are correlated with each other) affects how strongly correlated the total scale can be with other measures. A rough rule of thumb is that the reliability of a scale sets an upper bound for the correlation between that scale and others. Since these two subsets of Porosity are kind of ad-hoc (and definitely post-hoc) breakdowns of a pre-existing scale, this seems especially important to check here.

With this in mind, let’s check the reliability of the two subsets of the Porosity scale, and look at how these correlations with Spiritual Events stack up to the upper limit imposed by reliability… and just for fun I’ll throw in the correlations within each site as well.

The solid dots in these plots illustrate that the less personified Porosity items (rightmost panel) formed a generally much less reliable “scale” than the more personified Porosity items (center)—not surprising, since shorter scales are generally less reliable. This has the important consequence of forcing the correlation between the less personified items and the Spiritual Events scale in some sense to be weaker.

Given this, what I’m interested in is whether the distance between the cross-hatch (the correlation coefficient between that scale and Spirtual Events) and the solid dot (the reliability of that scale) differs across the panels. If this distance were greater for the less personified Porosity items, this would strengthen my belief that these items were meaningfully less strongly related to Spiritual Events - but that’s not really what I see here. Instead, if you think of reliability as an upper limit for correlation, the differences between correlations starts to look like an artifact of differences in reliability. (If anything, the reverse might be true - these items might be closer to the ceiling for correlation?)

KARA’S CURRENT TAKE-AWAY: Both subsets of Porosity items were correlated with Spiritual Events. These relationships were weaker for the less personified Porosity items, but I’m not sure these differences were meaningful—they might well be an artifact of underlying differences in reliability across the two subsets of Porosity items.

Question #4: Focusing on the purest porosity events from the Martha Story scale

Tanya’s question: Can we sum Martha Mary 1b, 1e, 2b, 2e, 3b, 3e and see whether these “pure” porosity events still predict spiritual events?

BOOKMARK

I’ll start by looking at correlations between the pure Martha Story items, the other Martha Story items and the Spiritual Events total score. I’ll also include the relevant correlations with the Martha Story total score as a point of reference.

__Correlations with Spiritual Events_:

• The pure subest of Martha Story items was moderately correlated with the Spiritual Events total score (r = 0.49, 95% CI: [NA, NA], p = NA).
• Numerically speaking, this correlation appears to be identical to the correlation between the other subset of SE items (#1-13) and Martha Story (r = 0.5, 95% CI: [NA, NA], p = NA).

However, is this a meaningful similarity? To repeat from Questions #1-2, above: an important thing to keep in mind is that the reliability of a scale (how strongly its items are correlated with each other) affects how strongly correlated the total scale can be with other measures. A rough rule of thumb is that the reliability of a scale sets an upper bound for the correlation between that scale and others. Since these two subsets of Martha Story are kind of ad-hoc (and definitely post-hoc) breakdowns of a pre-existing scale, this seems especially important to check here.

With this in mind, let’s check the reliability of the two subsets of the Martha Story scale, and look at how these correlations with Spiritual Events stack up to the upper limit imposed by reliability… and just for fun I’ll throw in the correlations within each site as well.

The solid dots in these plots illustrate that the pure Martha Story items (rightmost panel) and the other Martha Story items (center) formed roughly equally reliable “scales.” Thus, in principle, correlations with Spiritual Events could be equally strong for the two subsets of Martha Story items.

In this light, I feel pretty confidence saying that the “pure” Martha Story items were just as good at “predicting” Spiritual Events as were the other Martha Story items.

KARA’S CURRENT TAKE-AWAY: Both subsets of Martha Story items were equally correlated with Spiritual Events.

Question #5: Focusing religiosity & porosity

Tanya’s question: Are there highly religious people (as marked in demographics) who do not endorse porosity highly and the reverse? e.g., can we plot the how spiritual/religious against Martha Mary and also against Dulin?

To address this, I’ll plot self-reported religiosity against scores on both porosity measures.

We’re looking for two things here:

1. Participants who scored high on religiosity but low on porosity (i.e., points in the lower right corner).
2. People who scored low on religiosity but high on porosity (i.e., points in the upper left corner).

I’ve boxed off the regions that would contain (1) participants who scored 2 on religiosity and <= 25% of the total possible score for porosity, and (2) participants who scored -2 for religiosity and >= 75% of the total posible score for porosity. There are very few participants in these regions in any site.

Number of participants with unusual combinations of religiosity and porosity scores. 'High' corresponds to scores >= 25% of the total possible score; 'low' corresponds to scores <= 25% of the total possible score.

Site	High religiosity, low Porosity (Dulin)	High religiosity, low Martha Story	Low religiosity, high Porosity (Dulin)	Low religiosity, high Martha Story
US	2 (0.5%)	3 (0.7%)	0	0
Ghana	4 (1%)	6 (1.5%)	0	0
Thailand	0	1 (0.2%)	0	0
China	0	0	0	0
Vanuatu	0	1 (0.3%)	0	0

Rather than setting arbitrary cutoffs of +/-2 on religiosity and <=25% or >=75% of the total possible score for porosity, we could also look at “outliers” who scored a certain amount away from the mean on these measures. I’ll do this through standardization - first, collapsing across sites; then, within each site individually.

Parallel to above, I’ve boxed off the regions that would contain (1) participants who scored >= 1 standard deviation above the mean for religiosity and <= 1 standard deviation below the mean for porosity, and (2) participants who scored <= 1 standard deviation below the mean for religiosity and >= 1 standard deviation above the mean for porosity. There are very few participants in these regions in any site.

Number of participants with unusual combinations of religiosity and porosity scores (standardizing collapsing across sites). 'High' corresponds to scores >=1 standard deviation above the mean (collapsing across sites); 'low' corresponds to scores <= 1 standard deviation below the mean (collapsing across sites).

Site	High religiosity, low Porosity (Dulin)	High religiosity, low Martha Story	Low religiosity, high Porosity (Dulin)	Low religiosity, high Martha Story
US	0	1 (0.2%)	0	3 (0.7%)
Ghana	3 (0.8%)	2 (0.5%)	1 (0.2%)	1 (0.2%)
Thailand	0	0	0	0
China	0	0	0	1 (0.2%)
Vanuatu	0	0	0	0

What if we standardize within each site, such that we’re looking for participants who scored relatively higher/lower compared to other people in that site?

Parallel to the previous plot, I’ve boxed off the regions that would contain (1) participants who scored >= 1 standard deviation above the mean for that site for religiosity and <= 1 standard deviation below the mean for that site for porosity, and (2) participants who scored <= 1 standard deviation below the mean for that site for religiosity and >= 1 standard deviation above the mean for that site for porosity. There are very few participants in these regions in any site.

Number of participants with unusual combinations of religiosity and porosity scores (standardizing within each site). 'High' corresponds to scores >=1 standard deviation above the mean (for that site); 'low' corresponds to scores <= 1 standard deviation below the mean (for that site).

Site	High religiosity, low Porosity (Dulin)	High religiosity, low Martha Story	Low religiosity, high Porosity (Dulin)	Low religiosity, high Martha Story
US	0	0	1 (0.2%)	2 (0.5%)
Ghana	6 (1.5%)	6 (1.5%)	1 (0.2%)	2 (0.5%)
Thailand	0	0	0	0
China	2 (0.5%)	3 (0.8%)	1 (0.2%)	1 (0.2%)
Vanuatu	1 (0.3%)	2 (0.5%)	4 (1.1%)	2 (0.5%)

KARA’S CURRENT TAKE-AWAY: No matter how we look at it, there were just a handful of people (<2% in any site) who bucked the general trend that endorsements of porosity are positively related to self-reported religiosity.

Question #6: Focusing on highly religious people & spiritual experience

Tanya’s question: Plot highly religious people against the DSE and against the Spiritual Events scale?

NOTE: I’m assuming that this means to do something similar to Question #5, but I’m not sure if that was the intention.

To address this, I’ll plot self-reported religiosity against scores on both spiritual experience measures.

We’re looking for two things here:

1. Participants who scored high on religiosity but low on spiritual experience (i.e., points in the lower right corner).
2. People who scored low on religiosity but high on spiritual experience (i.e., points in the upper left corner).

I’ve boxed off the regions that would contain (1) participants who scored 2 on religiosity and <= 25% of the total possible score for spiritual experience, and (2) participants who scored -2 for religiosity and >= 75% of the total posible score for spiritual experience. There are very few participants in these regions in any site.

Number of participants with unusual combinations of religiosity and spiritual experience scores. 'High' corresponds to scores >= 25% of the total possible score; 'low' corresponds to scores <= 25% of the total possible score.

Site	High religiosity, low DSE	High religiosity, low SE	Low religiosity, high DSE	Low religiosity, high SE
US	0	6 (1.5%)	0	0
Ghana	0	10 (2.5%)	0	0
Thailand	1 (0.2%)	1 (0.2%)	0	0
China	0	0	0	0
Vanuatu	0	7 (1.9%)	0	0

Rather than setting arbitrary cutoffs of +/-2 on religiosity and <=25% or >=75% of the total possible score for spiritual experience, we could also look at “outliers” who scored a certain amount away from the mean on these measures. I’ll do this through standardization - first, collapsing across sites; then, within each site individually.

Parallel to above, I’ve boxed off the regions that would contain (1) participants who scored >= 1 standard deviation above the mean for religiosity and <= 1 standard deviation below the mean for spiritual experience, and (2) participants who scored <= 1 standard deviation below the mean for religiosity and >= 1 standard deviation above the mean for spiritual experience. There are virtually no participants in these regions in any site.

Number of participants with unusual combinations of religiosity and spiritual experience scores (standardizing collapsing across sites). 'High' corresponds to scores >=1 standard deviation above the mean (collapsing across sites); 'low' corresponds to scores <= 1 standard deviation below the mean (collapsing across sites).

Site	High religiosity, low DSE	High religiosity, low SE	Low religiosity, high DSE	Low religiosity, high SE
US	0	0	0	0
Ghana	0	2 (0.5%)	1 (0.2%)	0
Thailand	1 (0.2%)	0	0	0
China	0	0	0	0
Vanuatu	0	0	0	0

What if we standardize within each site, such that we’re looking for participants who scored relatively higher/lower compared to other people in that site?

Parallel to the previous plot, I’ve boxed off the regions that would contain (1) participants who scored >= 1 standard deviation above the mean for that site for religiosity and <= 1 standard deviation below the mean for that site for spiritual experience, and (2) participants who scored <= 1 standard deviation below the mean for that site for religiosity and >= 1 standard deviation above the mean for that site for spiritual experience. There are very few participants in these regions in any site.

Number of participants with unusual combinations of religiosity and spiritual experience scores (standardizing within each site). 'High' corresponds to scores >=1 standard deviation above the mean (for that site); 'low' corresponds to scores <= 1 standard deviation below the mean (for that site).

Site	High religiosity, low DSE	High religiosity, low SE	Low religiosity, high DSE	Low religiosity, high SE
US	0	0	0	0
Ghana	0	5 (1.2%)	2 (0.5%)	2 (0.5%)
Thailand	1 (0.2%)	0	0	0
China	1 (0.2%)	2 (0.5%)	1 (0.2%)	1 (0.2%)
Vanuatu	3 (0.8%)	2 (0.5%)	1 (0.3%)	3 (0.8%)

KARA’S CURRENT TAKE-AWAY: No matter how we look at it, there were just a handful of people (<3% in any site) who bucked the general trend that endorsements of spiritual experience are positively related to self-reported religiosity.